1. Technical Field
The present invention relates to optimization of functions and more particularly to a system and method for optimizing functions with a large number of parameters.
2. Description of the Related Art
Optimization of functions with a large number of parameters is very complex. A few techniques for function optimization include an exact computation of a Hessian matrix for Newton type of optimization methods. This type of computation is difficult since it requires processing around a squared number of parameters. This results in a slow speed and memory overload in many instances.
Hill-climbing algorithms require step sizes being defined for each iteration and also have a relatively slow convergence. Optimization algorithms that are based on approximated versions of Hessians are counter-intuitive and difficult to customize to specific tasks in which a learning ratio is needed to be controlled.
Therefore, there is a need for an optimization method and system for optimization for a large number of parameters that is fast, intuitive and easy to customize to special tasks.